<!DOCTYPE html>
<html>
	<head>
		<title>Voice Conversion Evaluation</title>
	</head>
	<body>
	<h1>Voice Conversion Evaluation</h1>
	<p>
		Thanks for doing this subjective evalation test! This is a subjective evaluation on voice conversion.
		You are supposed to evaluate the utterances converted by system1 and system2, while the source speech and reference speech
		are ground truths from source and target speakers respectively. Please fill in the Excel form attached to this web page.
		
	</p>
	
	<p>
		For naturalness mos evaluation, you should focus on the naturalness of the utterance from system1 and system2 and
		pay no attention to the speaker similarity with the reference speech. Score the utterances on a scale from
		1 (completely unnatural) to 5 (completely natural).
	</p>
	
	<p>
		For similarity evaluation, you should focus on the speaker similarity of the utterances to the reference speech and
		pay no attention to the naturalness. Choose system whose converted speech is more similar to the reference speech,
		1 for system1, 2 for system2, NP for no preference.
	</p>
	<p>
		Attention! The reference speech's linguistic content is not necessarily the same as the converted speech!
	</p>
	<p> Please send the filled Excel form to me via wechat or <a href = "mailto: lu-h17@mails.tsinghua.edu.cn">Email</a>. Thanks again!
	</p>
	
		<h2>Male-to-Female</h2>
		<table style="width:100%">
		  <tr>
			<th>index</th>
			<th>Source Speech</th>
			<th>Reference Speech</th> 
			<th>System 1</th>
			<th>System 2</th>
		  </tr>
		  <tr>
			<th>1.m2f1</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p225_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p225_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		  </table>
		  <p>We also need a small plastic snake and a big toy frog for the kids.</p>
		  <table style="width:100%">
		  <tr>
		  <th>2.m2f2</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p225_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p225_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>There is a new confidence about the club.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>3.m2f3</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p225_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p225_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>If they are not, the commission will take action.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>4.m2f4</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_383.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p225_383.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p225_383.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>It was a double tragedy for the family.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>5.m2f5</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_020.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_020.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p225_020.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p225_020.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>Many complicated ideas about the rainbow have been formed.</p>
		
		<h2>Female-to-Female</h2>
		<table style="width:100%">
		  <tr>
			<th>index</th>
			<th>Source Speech</th>
			<th>Target Speech</th> 
			<th>System 1</th>
			<th>System 2</th>
		  </tr>
		  <tr>
			<th>6.f2f1</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p225_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p225_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		  </table>
		  <p>We also need a small plastic snake and a big toy frog for the kids.</p>
		  <table style="width:100%">
		  <tr>
		  <th>7.f2f2</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p225_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p225_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>This is great news.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>8.f2f3</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_063.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_063.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p225_063.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p225_063.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>Together, we can make a difference.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>9.f2f4</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_214.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p225_214.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p225_214.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>He was outstanding today - the best player on the park.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>10.f2f5</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p225_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p225_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p225_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>He said he never saw it.</p>
		
		<h2>Male-to-Male</h2>
		<table style="width:100%">
		  <tr>
			<th>index</th>
			<th>Source Speech</th>
			<th>Target Speech</th> 
			<th>System 1</th>
			<th>System 2</th>
		  </tr>
		  <tr>
			<th>11.m2m1</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p315_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p315_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		  </table>
		  <p>We also need a small plastic snake and a big toy frog for the kids.</p>
		  <table style="width:100%">
		  <tr>
		  <th>12.m2m2</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p315_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p315_095.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>If they are not, the commission will take action.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>13.m2m3</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_078.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_078.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p315_078.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p315_078.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>We did not deserve to go through.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>14.m2m4</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_214.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_214.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p315_214.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p315_214.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>This is not a monument to the past.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>15.m2m5</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p363_003.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_003.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p363_p315_003.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p363_p315_003.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>Six spoons of fresh snow peas, five thick slabs of blue cheese, and maybe a snack for her brother Bob.</p>
		
		<h2>Female-to-Male</h2>
				<table style="width:100%">
		  <tr>
			<th>index</th>
			<th>Source Speech</th>
			<th>Target Speech</th> 
			<th>System 1</th>
			<th>System 2</th>
		  </tr>
		  <tr>
			<th>16.f2m1</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p315_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p315_004.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		  </table>
		  <p>We also need a small plastic snake and a big toy frog for the kids.</p>
		  <table style="width:100%">
		  <tr>
		  <th>17.f2m2</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p315_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p315_042.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>This is great news.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>18.f2m3</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_063.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p315_063.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p315_063.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>Together, we can make a difference.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>19.f2m4</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_383.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_023.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p315_383.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p315_383.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>I hadn't had that before.</p>
		
		<table style="width:100%">
		  <tr>
		  <th>20.f2m5</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/source/p340_001.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/target/p315_001.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th> 
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system1/p340_p315_001.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
			<th>
					<audio controls style="width: 200px;">
					  <source src="./wavs/system2/p340_p315_001.wav" type="audio/mpeg">
					Your browser does not support the audio element.
					</audio>
			</th>
		  </tr>
		</table>
		<p>Please call Stella.</p>
		
	</body>
</html>